Variational Approximations between Mean Field Theory and the Junction Tree Algorithm
نویسنده
چکیده
Recently, variational approximations such as the mean field approximation have received much interest. We extend the standard mean field method by using an approximating dis tribution that factorises into cluster poten tials. This includes undirected graphs, di rected acyclic graphs and junction trees. We derive generalised mean field equations to op timise the cluster potentials. We show that the method bridges the gap between the stan dard mean field approximation and the exact junction tree algorithm. In addition, we ad dress the problem of how to choose the struc ture and the free parameters of the approx imating distribution. From the generalised mean field equations we derive rules to sim plify the approximation in advance without affecting the potential accuracy of the model class. We also show how the method fits into some other variational approximations that are currently popular.
منابع مشابه
Approximate Inference in Credal Networks by Variational Mean Field Methods
Graph-theoretical representations for sets of probability measures (credal networks) generally display high complexity, and approximate inference seems to be a natural solution for large networks. This paper introduces a variational approach to approximate inference in credal networks: we show how to formulate mean field approximations using naive (fully factorized) and structured (tree-like) s...
متن کاملConditional mean field
Despite all the attention paid to variational methods based on sum-product message passing (loopy belief propagation, tree-reweighted sum-product), these methods are still bound to inference on a small set of probabilistic models. Mean field approximations have been applied to a broader set of problems, but the solutions are often poor. We propose a new class of conditionally-specified variatio...
متن کاملDynamic Trees: A Structured Variational Method Giving Efficient Propagation Rules
Dynamic trees are mixtures of tree struc tured belief networks. They solve some of the problems of fixed tree networks at the cost of making exact inference intractable. For this reason approximate methods such as sampling or mean field approaches have been used. However, mean field approximations assume a factorised distribution over node states. Such a distribution seems unlikely in the post...
متن کاملMixture Approximations to Bayesian Networks
Structure and parameters in a Bayesian network uniquely specify the probability distribution of the modeled domain. The locality of both structure and probabilistic information are the great benefits of Bayesian networks and require the modeler to only specify local information. On the other hand this locality of information might prevent the modeler —and even more any other person— from obtain...
متن کاملA Variational Mean-Field Theory for Sigmoidal Belief Networks
A variational derivation of Plefka's mean-field theory is presented. This theory is then applied to sigmoidal belief networks with the aid of further approximations. Empirical evaluation on small scale networks show that the proposed approximations are quite competitive.
متن کامل